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EXECUTIVE  SUMMARY 


Extensive  research  has  focused  on  the  behavioral  and 
organizational  aspects  of  operational  error  (OE)  occur¬ 
rence.  While  recognizing  that  the  human  component  of 
OEs  is  extremely  important,  it  is  equally  important  to 
examine  contextual  and  environmental  factors.  In  this 
study,  we  analyzed  the  extent  to  which  controller  ratings 
of  static  and  dynamic  sector  complexity  factors  related 
to  the  occurrence  of  OEs  at  the  Indianapolis  air  route 
traffic  control  center  (ZID).  OE  information  was  derived 
from  final  reports  for  247  errors  that  occurred  between 
1/15/2001  and  5/28/2005.  Thirty-six  ZID  volunteers 
(32  controllers  and  4  operational  supervisors)  rated  the 
importance  of  22  static  and  dynamic  complexity  factors 
for  each  sector  on  which  they  were  certified. 

Principal  Components  Analysis  (PCA)  of  sector  com¬ 
plexity  ratings  produced  four  components  that  accounted 
for  approximately  62%  of  the  variance  in  the  dataset.  The 
basic  theory  behind  PCA  is  that  variables  cluster  together 
into  “components”  that  reflect  underlying  dimensions  in 
the  data.  In  this  PCA,  the  pattern  of  variables  associated 
with  Component  1  described  climbing  and  descending 
aircraft  in  the  vicinity  of  major  airports.  Component  2 
variables  involved  ATC  services  provided  to  non-towered 
airports.  Component  3  comprised  variables  associated  with 
military  operations  and  special  use  airspace.  Component 
4  described  the  effects  of  inclement  weather  on  ATC 
operations.  These  results  were  comparable  in  many  ways 
to  the  PCA  of  sector  complexity  factors  at  the  Atlanta 
ARTCC  conducted  by  Rodgers,  Mogford,  and  Mogford 
(1998).  Specifically,  Components  1  (major  airports)  and 
3  (military  activity  and  Special  Use  Areas  [SUAs])  of  the 
two  analyses  were  strikingly  similar,  suggesting  that  these 
dimensions  may  be  common  to  more  than  one  facility. 

A  multiple  regression  analysis  was  conducted  to  examine 
the  relationship  between  the  four  dimensions  revealed  by 
the  PCA  and  the  number  of  OEs.  Only  Component  1 
(major  airports)  and  Component  2  (non-towered  airports) 
explained  a  significant  amount  of  the  variance  in  OEs  in 


the  ZID  sectors  (R  =  .78,  R2  =  .61).  Component  1  was 
positively  associated  with  the  number  of  OEs  (i.e.,  higher 
scores  were  related  to  a  higher  number  of  OEs),  whereas 
Component  2  had  a  negative  relationship  (higher  scores 
were  related  to  fewer  OEs).  The  relationship  between 
Component  2  and  the  incidence  of  OEs  reminds  us  that 
sector  complexity  does  not  always  produce  a  negative 
outcome.  Indeed,  a  certain  degree  or  type  of  complex¬ 
ity  may  actually  be  associated  with  a  reduction  in  the 
number  of  OEs.  The  fact  that  Component  3  failed  to 
contribute  significantly  to  the  prediction  of  OEs  in  this 
analysis  does  not  mean  that  military  airspace  or  SUAs 
do  not  make  a  sector  more  difficult  to  work  or  increase 
the  likelihood  of  an  OE.  It  simply  means  that  subjective 
ratings  of  these  factors  failed  to  predict  OEs.  Similarly,  the 
inability  of  Component  4  to  contribute  significantly  to 
the  regression  model  may  reflect  the  intermittent  nature 
of  this  dynamic  event,  or  it  may  simply  be  an  artifact  of 
the  way  the  variable  was  measured.  In  other  words,  the 
presence  of  inclement  weather  might  be  highly  correlated 
with  the  occurrence  of  OEs  but  the  component  scores 
based  on  subjective  ratings  of  variables  associated  with 
inclement  weather  were  not. 

We  believe  it  is  imprudent  to  make  strong  recom¬ 
mendations  based  on  the  results  of  analysis  of  subjective 
ratings.  Practical  prediction  models  must  be  calculated 
from  objective  measures  because  the  actual  characteristics 
of  the  sectors  must  be  addressed  when  developing  strate¬ 
gies  to  reduce  OEs.  Elowever,  these  results  represent  a 
necessary  and  important  step  toward  understanding  how 
static  and  dynamic  sector  characteristics  combine  to  cre¬ 
ate  sector  complexity,  and  how  that  complexity  relates  to 
the  occurrence  of  OEs.  Subjective  information  about  the 
importance  of  complexity  factors  will  guide  our  choice 
of  objective  measures  in  future  analyses  and  may  also 
be  used  to  weight  their  importance,  thus  enabling  us  to 
make  recommendations  for  reasonable,  effective  changes 
to  the  current  system. 


iii 


Relationship  of  Complexity  Factor  Ratings 
With  Operational  Errors 


The  Federal  Aviation  Administration  (FAA)  is  cur¬ 
rently  implementing  strategic  safety  initiatives  aimed  at 
reducing  operational  error  rates  (FAA,  2005).  Toward 
that  goal,  a  considerable  amount  of  research  has  focused 
on  behavioral  and  organizational  aspects  of  operational 
error  (OE)  occurrence.  While  recognizing  that  the  hu¬ 
man  component  of  OEs  is  extremely  important,  it  is  also 
important  to  remember  that  air  traffic  controllers  do  not 
operate  in  a  void.  Logic  dictates  that  environmental  and 
contextual  factors  contribute  to  the  development  of  at 
least  a  portion  of  these  errors.  Otherwise,  the  frequency 
of  OEs  would  be  relatively  equal  in  all  sectors.  This 
simply  is  not  the  case.  Some  sectors  are  more  prone  to 
OEs  than  others. 

The  idea  that  sector  characteristics  might  contribute 
to  the  occurrence  of  OEs  is  not  new.  Environmental 
and  contextual  factors  affecting  controller  workload  and 
performance  -  often  referred  to  as  sector  complexity 
-  have  been  the  focus  of  numerous  studies  (e.g.,  Arad, 
1964;  Arad,  Golden,  Grambart,  Mayfield,  &  Van  Saun, 
1963;  Buckley,  O’Connor,  &  Beebe,  1969;  Davis,  Da- 
naher,  &  Fischl,  1963;  Grossberg,  1989;  Hurst  &  Rose, 
1978;  Kirwan,  Scaife,  &  Kennedy,  2001;  Kopardekar 
&  Magyarits,  2003;  Laudeman,  Shelden,  Branstrom, 
&  Brasil,  1998;  Masalonis,  Callaham,  &  Wanke,  2003; 
Mogford,  Murphy  &  Guttman,  1 994;  Rodgers,  Mogford, 
&Mogford,  1998;  Schmidt,  1976;  Stager,  Hameluck,  & 
Jubis,  1989;  Stein,  1985).  Even  so,  a  single  set  of  reliable 
general  complexity  factors  has  remained  elusive.  This  may 
be  partially  due  to  the  complicated  interaction  between 
static  and  dynamic  sector  characteristics.  Static  sector 
characteristics  are  usually  related  to  airspace  design  and 
change  infrequently  or  not  at  all.  Dynamic  sector  charac¬ 
teristics  are  those  that  fluctuate,  such  as  traffic  volume  or 
weather.  Mogford,  Guttman,  Morrow,  and  Kopardekar 
(1995)  observed  that  “a  given  level  of  traffic  density  and 
aircraft  characteristics  may  create  more  or  less  complexity 
depending  on  the  structure  of  the  sector”  (p.  3).  Buckley, 
DeBarysche,  Hitchner,  and  Kohn  (1983)  concluded  that 
traffic  characteristics  and  sector  geometry  were  “important 
factors  in  determining  the  results  which  will  occur  in  a 
given  experiment,  but  they  interact  in  a  complex  way. 
The  nature  and  extent  of  this  interaction  depends  upon 
the  measures  involved”  (p.  73). 

Perhaps  one  of  the  greatest  challenges  to  the  study 
of  environmental  factors  in  the  development  of  OEs  is 
that  sectors  are  almost  as  unique  as  the  people  who  work 


them.  For  example,  Grossberg  (1989)  found  that  the 
highest-rated  complexity  factors  in  the  Chicago  air  route 
traffic  control  center  (ARTCC)  were  control  adjustments 
involved  in  merging  and  spacing  aircraft,  climbing  and 
descending  aircraft,  mix  of  aircraft  types,  frequent  coor¬ 
dination,  and  amount  of  traffic.  An  index  based  on  these 
factors  was  significantly  correlated  with  the  number  of 
OEs.  In  the  Jacksonville  airspace,  Mogford  and  coworkers 
( 1 994)  found  complex  routings,  spacing  and  sequencing 
for  departures  and  arrivals,  and  frequency  congestion 
to  be  most  predictive  of  a  subjective  complexity  index. 
Comparing  their  results  with  Grossberg’s,  they  concluded 
that  complexity  factors  that  were  salient  in  one  facility 
might  not  be  applicable  to  another.  Although  this  observa¬ 
tion  may  be  valid,  another  interpretation  is  that  different 
complexity  factors  were  found  to  be  predictive  because 
the  two  studies  used  different  criteria.  In  other  words, 
numbers  of  OEs  and  the  value  of  a  subjective  complexity 
index  may  not  be  comparable  criterion  measures. 

This  brings  us  to  an  important  point:  It  is  extremely 
difficult  to  compare  the  results  of  the  studies  cited  be¬ 
cause  of  the  variety  of  methods  and  measures  used  to 
assess  complexity.  Some  of  the  studies  compared  sec¬ 
tor  complexity  factors  with  OEs,  some  with  subjective 
workload  measures,  and  some  with  subjective  complexity 
ratings.  On  the  basis  of  a  comprehensive  review  of  the 
literature  that  spanned  more  than  40  years  of  research  and 
identified  in  excess  of  100  complexity  factors,  Hilburn 
(2004)  concluded  that  “despite  the  breadth  and  depth 
of  previous  work  done  into  identifying  ATC  complexity 
factors,  a  good  deal  of  work  remains.  Nobody,  it  seems, 
has  yet  managed  to  construct  a  valid  and  reliable  model 
of  ATC  complexity  that  [1]  moves  substantially  beyond 
the  predictive  value  of  simple  traffic  density  alone,  and 
[2]  is  sufficiently  context-free”  (p.  vi).  In  his  conclusions, 
Hilburn  makes  an  excellent  case  for  the  use  of  non-linear 
models  of  complexity  to  circumvent  the  context-specific 
shortcomings  of  linear  models.  The  only  drawback  to 
this  argument  is  that  it  fails  to  recognize  that  sometimes 
the  context  is  the  factor  of  interest.  This  is  not  to  imply 
that  Hilburn  was  ignoring  the  importance  of  context 
or  the  inevitability  of  contextual  influences  in  airspace 
complexity.  Rather,  it  emphasizes  the  fact  that  develop¬ 
ment  of  a  “context-free”  model  is  not  always  a  desirable 
goal  in  research. 

In  spite  of  the  “embarrassment  of  riches”  represented 
by  the  literature,  one  precept  is  evident:  It  is  extremely 
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important  to  compare  complexity  factors  in  as  many 
environments  as  possible.  The  Sector  Characteristics 
and  Operational  Errors  (SCOpE)  project  is  an  exten¬ 
sion  of  a  study  conducted  by  Rodgers,  Mogford,  and 
Mogford  (1998)  that  examined  the  relationship  between 
sector  complexity  factors  and  the  occurrence  of  OEs  at 
the  Atlanta  ARTCC  (ZTL).  Specifically,  the  SCOpE 
project  was  initiated  to  compare  and  contrast  the  results 
of  selected  analyses  from  the  1998  study  with  similar 
analyses  conducted  using  data  from  the  Indianapolis 
ARTCC  (ZID).  The  methodology  of  developing  a  re¬ 
gression  model  at  one  facility  and  applying  the  derived 
regression  weights  to  anotherfacilityhas  met  with  limited 
success  (e.g.,  Laudeman  et  al.,  1998;  Masalonis  et  ah, 
2003).  The  advantage  of  the  SCOpE  paradigm  is  that  it 
employs  discrete  models,  thus  enabling  us  to  collect  a  set 
of  general  factors  (i.e.,  factors  that  may  reliably  predict 
OEs  at  more  than  one  facility)  while  documenting  dif¬ 
ferences  between  facilities.  After  all,  facility  differences 
often  represent  important  environmental  and  contextual 
elements  as  well. 

The  first  phase  of  this  project  (Goldman,  Manning, 
&  Pfleiderer,  2006)  compared  a  set  of  ZID  static  sector 
characteristics  with  those  identified  by  Rodgers  et  al. 
(1998)  at  ZTL.  With  some  exceptions,  many  of  the  static 
environmental  and  contextual  factors  that  predicted  OEs 
at  ZTL  also  predicted  the  occurrence  of  OEs  at  ZID.  In 
both  studies,  sector  altitude  strata,  sector  size,  and  number 
of  major  airports  explained  a  significant  proportion  of 
the  variance  in  the  number  of  OEs  per  sector.  Elowever, 
some  factors  that  were  significantly  correlated  with  OEs 
at  ZTL  failed  to  predict  them  in  the  ZID  sample. 

The  second  phase  of  the  SCOpE  project  considers 
the  relationship  between  a  set  of  subjective  static  and 
dynamic  complexity  factor  ratings  and  ZID  OEs.  In 
their  analysis  of  sector  characteristics  and  OEs,  Rodgers 
et  al.  (1998)  collected  subjective  complexity  ratings  and 
combined  them  with  obj  ective  static  sector  characteristics 
in  a  Principal  Components  Analysis  (PCA)  to  identify 
and  describe  the  dimensions  represented  by  these  dif¬ 
ferent  types  of  variables.  In  the  present  study,  subjective 
complexity  ratings  provided  by  ZID  controllers  will  be 
examined  in  a  series  of  discrete  analyses  to  evaluate  their 
relationship  with  OEs  at  ZID. 

Method 

Participants 

Participants  were  37  volunteers  from  ZID.  Of  these, 
32  were  Certified  Professional  Controllers  (CPCs),  4 
were  operations  supervisors,  and  1  was  a  developmental 
controller  who  had  completed  Radar  Associate  training 


on  all  sectors  in  his  area  of  specialization  but  was  not  yet 
certified  on  the  corresponding  radar  positions.  To  guaran¬ 
tee  that  all  participants  were  treated  fairly  and  ethically, 
the  experimental  protocol  and  materials  were  cleared 
through  the  FAA’s  institutional  review  board.  Treatment 
of  participants  also  met  with  guidelines  established  by 
the  American  Psychological  Association.  Volunteers  were 
assured  complete  anonymity  and  reminded  of  their  right 
to  terminate  participation  at  any  time. 

The  mean  age  of  the  volunteer  participants  was  42 
years  (SD  =  6  years).  Participants  had  been  certified  to 
control  traffic  for  an  average  of  15  years  (SD  =  7  years), 
had  been  working  at  an  ARTCC  facility  for  a  mean  of 
17  years  (SD  =  7  years),  and  had  been  working  at  their 
current  facility  for  an  average  of  16  years  (SD  =  8  years). 
Four  had  previous  experience  in  the  Terminal  Radar  Ap¬ 
proach  Control  (TRACON)  environment,  and  six  had 
previously  worked  at  an  Airport  Traffic  Control  Tower 
(ATCT) .  ZID  is  divided  into  seven  areas  of  specialization, 
each  comprising  either  five  or  six  sectors.  All  areas  were 
reasonably  well  represented  by  the  sample  of  volunteer 
controllers  and  supervisors. 

Materials 

Complexity  Factor  Questionnaire  (Complexity-Q). 
“Complexity-Q”  refers  to  an  automated  experimental 
protocol  software  program  and  the  questionnaire  it  was 
designed  to  administer.  The  Complexity-Q  program  is 
divided  into  four  sections  (i.e.,  Work  Experience,  Tuto¬ 
rial,  Demonstration,  and  Questionnaire).  Each  section  is 
described  separately  in  the  following  paragraphs. 

The  Work  Experience  section  recorded  biographical 
data  about  participants’  work  experience,  collected  infor¬ 
mation  about  the  sectors  on  which  they  were  certified, 
and  generated  a  random-ordered  list  of  sectors  to  be 
included  in  the  questionnaire  (based  on  input  from  the 
participant).  It  also  randomized  the  presentation  order 
of  the  complexity  factor  list  and  recorded  the  order  of 
both  lists  in  the  output. 

The  Tutorial  section  consisted  of  a  Microsoft  Pow¬ 
erPoint  slide  presentation  (automatically  opened  by  the 
Complexity-Q  program)  that  explained  the  purpose  of 
the  study,  provided  participants  with  an  operational  defi¬ 
nition  of  sector  complexity  (i.e.,  “the  static  and  dynamic 
characteristics  that  increase  the  level  of  difficulty  involved 
in  working  traffic  in  a  sector”),  familiarized  them  with 
the  basic  structure  of  the  questionnaire,  and  provided 
instructions  about  the  functionality  of  interface  elements 
(e.g.,  buttons,  sliders,  and  bars).  The  Demonstration 
section  was  an  extension  of  the  Tutorial  that  provided 
participants  with  an  opportunity  to  practice  using  the 
elements  described  therein. 
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The  Complexity  Factor  Questionnaire  followed  the 
same  basic  structure  for  each  sector  on  which  the  partici¬ 
pants  were  certified.  They  were  asked  to  provide  a  general 
“Complexity  Rating”  for  a  sector  using  a  slider  object 
with  an  underlying  scale  ranging  from  0  to  100.  The  end 
points  of  the  slider  were  labeled  “Low”  and  “High”  with 
visual  anchors  set  at  10-point  intervals.  However,  the 
slider  was  not  restricted  to  these  anchors,  thus  affording 
raters  maximum  response  flexibility. 

Once  the  participants  entered  an  overall  complexity 
rating,  they  were  presented  sequentially  with  a  series  of 
22  complexity  factors  and  asked  to  indicate  the  level  of 
influence  each  factor  had  on  the  complexity  of  the  sector. 
The  “Factor  Rating”  was  made  using  the  same  slider  and 
scale  as  the  general  complexity  rating.  If  the  participants 
were  unsure  about  the  meaning  of  the  factor,  a  detailed 
description  could  be  obtained  by  moving  the  mouse 
over  the  “Complexity  Factor”  label.  The  list  of  factors 
and  their  descriptions  was  initially  derived  from  the  19 
complexity  factors  identified  by  Mogford  et  al.  (1994). 
Subject  Matter  Experts  (SMEs)  from  the  facility  and  the 
FAA  Academy  provided  additional  factors  prior  to  data 
collection.  The  Complexity-Q  factors  and  their  descrip¬ 
tions  are  provided  in  Table  1.  Note  that  only  two  extra 
factors  were  added  by  the  SMEs,  yet  there  are  22  factors 
in  the  list.  The  “mix  of  aircraft  with  different  performance 
characteristics”  and  “VFR  versus  IFR  traffic”  factors  were 
combined  in  the  original  list  but  were  separated  into  two 
distinct  factors  for  this  study. 

After  participants  entered  factor  ratings  for  all  22 
factors,  they  were  given  the  opportunity  to  enter  any 
complexity  factors  they  believed  were  not  included  in 
the  list.  If  a  participant  added  a  complexity  factor,  the 
new  factor  was  added  to  that  particular  participant’s  list 
for  all  subsequent  sectors.  Thus,  whenever  participants 
entered  a  new  complexity  factor  to  the  list,  they  had  to 
rate  that  factor  for  all  remaining  sectors. 

Procedure 

Testing  took  place  from  6/ 13/2005  to  6/17/2005  in  a 
classroom  atZID.The  Complexity-Qautomated  protocol 
was  administered  on  laptop  computers  arranged  around 
a  large  table  to  provide  participants  with  as  much  privacy 
as  possible.  Participants  were  first  given  informed  consent 
forms  to  read  and  sign.  Once  their  written  consent  was 
obtained,  they  were  shown  the  basic  structure  of  the 
Complexity-Q  interface,  and  the  “Work  Experience” 
section  was  brought  up  on  the  screen.  Participants  were 
requested  to  complete  this  section  and  then  move  through 
all  subsequent  sections  in  the  order  they  appeared  on 
the  main  interface  (i.e.,  Tutorial,  Demonstration,  and 
Questionnaire).  They  were  encouraged  to  ask  questions 
about  the  interface  or  content  of  the  Complexity-Qat  any 


time  during  the  automated  protocol.  Most  participants 
completed  the  protocol  in  40  minutes. 

Measures 

In  addition  to  subjective  Complexity-Q  factor  ratings 
provided  by  controllers,  OE  and  sector  characteristics 
information  were  collected  from  data  provided  by  facil¬ 
ity  management.  The  following  sections  describe  these 
variables  and  their  sources. 

Operational  Error  ( OE)  Data.  The  OE  database  con¬ 
sisted  of  information  extracted  from  electronic  records 
of  the  Final  Operational  Error/Deviation  Report  (FAA 
Form  7210-3)  for  247  OEs  occurring  in  ZID  airspace 
from  1/15/2001  through  5/28/2005.  Variables  obtained 
from  the  final  OE  reports  included  the  date  and  time 
of  the  OE  and  the  number  of  controlled  aircraft  in  the 
sector  at  the  time  the  OE  occurred.  OEs  were  tallied  for 
each  sector  in  the  ZID  airspace. 

Sector  Characteristics  Data.  Sector  altitude  strata  (super 
high-,  high-,  intermediate  high-,  intermediate-,  or  low- 
altitude)  were  obtained  from  the  facility’s  Adaptation 
Control  Environmental  System  (ACES)  sector  description 
file  and  verified  with  sector  maps.  The  number  of  associ¬ 
ated  airports  and  the  number  of  airports  for  which  the 
sector  provided  approach  services  were  derived  from  sector 
descriptions  included  in  the  center’s  Standard  Operational 
Procedures  (SOPs).  Staff  from  the  facility’s  airspace  office 
clarified  and  augmented  this  information. 

Results  and  Discussion 

Descriptive  Statistics 

The  sample  consisted  of  181  complexity  ratings 
provided  by  CPCs  (n  =  1 69)  and  operations  supervisors 
(ji  =  12).  Prior  to  the  analysis,  sector-by-sector  compari¬ 
sons  were  made  between  the  mean  ratings  provided  by 
CPCs  and  those  provided  by  supervisors  to  determine 
whether  the  two  sets  of  observations  were  comparable. 
On  the  average,  supervisors’  ratings  were  less  than  two 
standard  deviations  from  those  of  controllers,  indicating 
that  supervisor  and  CPC  ratings  were  similar  enough  to 
constitute  a  homogenous  sample.  Conversely,  data  from 
the  single  developmental  controller  were  excluded  from  the 
analysis.  Supervisors  and  CPCs  had  experience  working 
both  the  Radar  Associate  and  Radar  positions,  whereas 
the  developmental  controller  did  not.  This  difference  in 
experience  gave  rise  to  a  discernable  pattern  of  rating  dif¬ 
ferences,  suggesting  that  the  developmental  was  sampled 
from  a  different  population.  Table  2  lists  descriptive  sta¬ 
tistics  for  the  Complexity-Qratings.  Though  many  of  the 
distributions  approximated  normality,  there  were  some 
notable  exceptions.  The  distribution  of  the  VFR  versus 
IFR  traffic,  Shelves/Tunnels,  Foreign  aircraft/pilots,  and 
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Table  1.  Complexity-Q  Complexity  Factors  and  Descriptions* 


Complexity  Factor 

Description 

Climbing  and  descending  traffic 

Climbing  and  descending  aircraft  are  those  that  are  transitioning 
altitudes,  including  departure  and  arrival  traffic,  aircraft  that  require 
different  altitudes  to  alleviate  conflictions  due  to  crossing  traffic  or 
other  problems,  and  aircraft  requesting  altitude  changes  due  to 
turbulence,  pilot  preference,  etc. 

Mix  of  aircraft  with  different 
performance  characteristics 

Extent  to  which  the  mix  of  props,  turboprops,  and  jets  impacts  the 
controller. 

VFR  versus  IFR  traffic 

Extent  to  which  differences  in  controlling  VFR  and  IFR  traffic,  or  VFR 
pilots  encountering  IFR  conditions  impacts  controller  workload. 

Number  of  intersecting  aircraft  flight 
paths 

The  number  of  converging  flight  paths  due  to  airways,  arrival  routes, 
frequent  requests  for  direct  routings;  number  of  airways  coming  into 
same  NAVAID;  number  of  routes  converging  on  a  STAR,  etc. 

Number  of  multiple  functions  controller 
must  perform 

Set  of  related  tasks  or  services  required  in  this  sector  (e.g.,  approach 
control,  terminal  feeder,  en  route,  and  in-trail  spacing). 

Traffic  volume 

Extent  to  which  the  number  of  aircraft  relative  to  the  amount  of 
available  airspace  impacts  the  controller. 

Amount  of  military  or  other  special 
traffic 

Number  of  special  missions  (e.g.,  military,  NASA,  flight  inspection, 
Lifeguard). 

Number  of  required  procedures  that 
must  be  performed  (i.e.,  crossing 
restrictions  in  LOAs) 

A  group  of  tasks,  or  a  specific  task,  required  by  regulation  or 
direction.  A  procedure  mandates  controller  actions  and  must  be 
performed  regardless  of  other  required  tasks. 

Amount  of  coordination/  interfacing 
required 

Coordination  with  adjacent  sectors,  approach  control,  other  en  route 
centers,  military  facilities,  etc. 

Major  airports  (inside  and  outside 
sector  boundaries)  that  might  influence 
the  number  of  procedures  used,  etc. 

Extent  to  which  the  controller's  work  is  affected  by  the  concentration 
of  flights  into  one  area  due  to  the  orientation  of  the  sector  relative  to 
one  or  more  major  airports. 

Extent  operations  are  affected  by 
weather 

Presence  of  weather  conditions  that  necessitate  requests  for 
deviations,  route  changes,  etc. 

Relative  frequency  of  complex  routings 

Frequency  of  aircraft  that  are  not  on  a  published  route  structure,  such 
as  vectors  or  direct  routings,  etc. 

Special  Use  Areas  (Restricted  areas, 
warning  areas,  and  military  operating 
areas)  and  their  associated  activities 

Extent  to  which  SUA  reduces  the  amount  of  airspace  for  non¬ 
participating  aircraft,  create  obstructions  to  flight  routes,  increase  the 
likelihood  of  conflictions  due  to  reroutes,  create  situations  requiring 
special  handling  and  monitoring,  etc. 

Size  of  sector  airspace 

The  volume  of  airspace  contained  within  the  lateral  and  horizontal 
boundaries  of  the  sector  and  the  extent  to  which  it  impacts  the 
controllers'  ability  to  handle  traffic  volume,  deal  with  special 
conditions  (e.g.,  weather),  and  conflict  resolution. 

Requirement  for  longitudinal  spacing/ 
sequencing 

Combining  aircraft  from  several  streams  into  one  stream. 

Adequacy  of  radio/radar  coverage 

Radio:  Extent  to  which  insufficient  radio  coverage  results  in  the  use  of 
alternate  communication  techniques,  such  as  pilot-to-pilot  relays. 

Radar:  Extent  to  which  lack  of  radar  results  in  use  of  non-radar 
procedures. 

Amount  of  radio  frequency  congestion 

Extent  to  which  radio  frequency  congestion  limits  the  controller's 
ability  to  utilize  the  frequency  for  issuing  instructions  to  aircraft. 

Traffic  Management  Initiatives 

Extent  to  which  Traffic  Management  Initiatives  impact  operations 
(e.g.,  miles  that  must  be  made  up,  vectoring  required  to  meet 
initiatives). 

Terrain/Obstructions 

Extent  to  which  the  terrain/obstructions  (e.g.,  mountainous  areas) 
adds  complexity. 

Shelves/Tunnels 

Extent  to  which  shelves  and/or  tunnels  add  to  complexity  of  the 
sector  (e.g.,  are  the  lateral  boundaries  of  the  sectors  above  or  below 
aligned  or  are  they  different?). 

OForeign  aircraft/pilots  with  English  as  a 
second  language 

Extent  to  which  communications  with  foreign  aircraft/pilots  with 

English  as  a  second  language  increases  the  difficulty  of 
communications. 

ONon-towered  airports 

Extent  to  which  providing  service  to  non-towered  airports  increases 
the  complexity  of  the  sector. 

*  Complexity  factors  and  descriptions  adapted  from  Mogford  et  al.  (1994)  except  where  indicated  (0) 
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Table  2.  Complexity-Q  Descriptive  Statistics  (N  =  181) 


Variable 

Mean 

SD 

Skew.1 

Kurtosis2 

Climbing  and  descending  traffic 

77.71 

20.94 

-0.86 

0.00 

Mix  of  aircraft  types 

63.75 

22.22 

-0.62 

0.30 

VFR  versus  IFR  traffic 

17.75 

28.15 

1.28 

0.08 

Number  of  intersecting  flight  paths 

70.94 

23.26 

-0.79 

-0.02 

Number  of  multiple  functions 

68.92 

21.18 

-0.64 

0.05 

Traffic  volume 

73.38 

21.14 

-0.82 

0.29 

Amount  of  military  or  other  special  traffic 

35.66 

25.59 

0.49 

-0.73 

Number  of  required  procedures 

69.67 

22.43 

-0.71 

0.11 

Coordination/  interfacing  reguired 

68.75 

21.82 

-0.61 

-0.08 

Number  of  major  airports 

68.19 

26.53 

-0.84 

-0.17 

Extent  operations  are  affected  by  weather 

75.52 

22.71 

-0.90 

0.10 

Relative  frequency  of  complex  routings 

48.75 

27.15 

-0.04 

-1.04 

Special  Use  Areas 

48.06 

29.09 

0.06 

-1.15 

Size  of  sector  airspace 

52.20 

27.17 

-0.11 

-0.87 

Longitudinal  spacing/  sequencing 

72.20 

25.44 

-1.02 

0.37 

Adequacy  of  radio/radar  coverage 

34.03 

32.49 

0.56 

-1.01 

Amount  of  radio  frequency  congestion 

60.50 

25.99 

-0.55 

-0.51 

Traffic  Management  Initiatives 

70.56 

27.40 

-0.90 

-0.08 

Terrain/Obstructions 

11.85 

23.98 

1.97 

2.54 

Shelves/Tunnels 

24.68 

30.84 

1.06 

-0.11 

Foreign  aircraft/pilots 

18.22 

19.93 

1.46 

1.43 

Non-towered  airports 

16.66 

29.24 

1.55 

0.86 

’ SE  Skew.  =  .181  2SE  Kurt.  =  .359 


Non-towered  airports  complexity  ratings  all  had  extreme 
positive  skews.  The  distribution  of  Terrain/Obstructions 
ratings  was  both  positively  skewed  and  leptokurtotic. 
Such  deviations  are  understandable,  considering  that 
these  complexity  factors  only  apply  to  some  sectors  (e.g., 
low- altitude  sectors,  sectors  with  shelves  or  tunnels). 
On  the  other  hand,  the  Extent  operations  are  affected 
by  weather  and  Requirements  for  longitudinal  spacing/ 
sequencing  were  given  high  ratings  in  almost  every  sector. 
Consequently,  the  distributions  of  these  variables  were 
significantly  negatively  skewed. 

Given  the  scale  of  some  of  the  deviations,  it  is  comfort¬ 
ing  to  know  that  assumptions  regarding  normality  are 
not  required  when  PCA  is  used  descriptively.  However, 
it  is  important  to  remember  that  PCA  is  sensitive  to  the 
magnitudes  of  correlations.  To  the  extent  that  normality 
fails,  the  solution  may  be  degraded  (Tabachnick  &  Fidell, 
1989).  Table  3  contains  a  Pearson’s  correlation  matrix  of 
all  the  Complexity-Q  factors.  Note  that  variables  with 
non-normal  distributions  achieved  a  significant  degree  of 
association  with  several  others  in  the  dataset,  suggesting 
that  the  deviations  were  not  severe  enough  to  prevent  a 
satisfactory  PCA  solution. 


Principal  Components  Analysis 

PCA  is  a  statistical  technique  often  used  to  describe 
relationships  between  complex  sets  of  variables.  Compo¬ 
nents  extracted  by  PCA  contribute  to  our  understand¬ 
ing  of  a  phenomenon  by  consolidating  variables  into 
parsimonious  groups.  In  orthogonal  rotation,  loadings 
represent  the  correlations  between  a  variable  and  a  com¬ 
ponent.  Variables  with  stronger  loadings  are  generally 
considered  to  be  more  representative  of  a  component’s 
underlying  processes.  Thus,  we  can  use  PCA  not  only  to 
identify  the  number  and  nature  of  unique  dimensions 
described  by  the  Complexity-Q  factors  but  also  to  what 
extent  each  variable  relates  to  them.  More  importantly, 
we  can  use  component  scores  output  from  the  PCA  to 
analyze  the  relationship  between  the  complexity  factors 
and  the  number  of  OEs  in  each  sector. 

For  the  Complexity-Q  analyses,  the  number  of  sec¬ 
tors  was  reduced  from  40  to  37  due  to  the  combination 
of  some  sectors  deemed  appropriate  by  ZID  personnel. 
With  only  37  sectors  in  the  sample  and  22  complexity 
factors,  the  case-to-predictor  ratio  is  unacceptable  for  most 
multivariate  analyses.  Component  scores  (computed  by 
weighting  variable  scores  using  regression-like  coefficients) 
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Table  3.  Correlation  Matrix  of  Complexity-Q  Factors  (N  =  181) 
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can  be  substituted  for  individual  Complexity-Q  ratings, 
thereby  reducing  the  number  of  predictors  without  losing 
information  about  their  interrelationships. 

Prior  to  the  analysis,  the  Kaiser-Meyer-Olkin  (KMO) 
measure  of  sampling  adequacy  was  examined  to  test 
whether  partial  correlations  among  the  variables  were 
small.  KMO  values  of  .6  and  above  are  required  for  a 
good  solution.  A  KMO  of  .87  was  produced  by  the  set  of 
variables  selected.  A  PCA  with  Varimax  rotation  converged 
in  nine  iterations  and  produced  four  components  with 
eigenvalues  >  1 .  These  components  accounted  for  approxi¬ 
mately  62%  of  the  variance  in  the  dataset.  As  shown  in  the 
rotated  component  matrix  in  Table  4,  all  but  one  of  the 
variables  had  a  loading  of  .50  or  greater  with  at  least  one 
of  the  components.  Only  Relative  frequency  of  complex 
routings  failed  to  demonstrate  a  strong  relationship  with 
any  one  dimension,  with  respective  loadings  of  .26,  -.35, 
.35,  and  .39  on  the  four  extracted  components. 

Component  1  had  an  eigenvalue  of  4.93  and  ac¬ 
counted  for  approximately  22%  of  the  variance  in  the 
dataset.  The  variables  with  high  loadings  on  this  com¬ 
ponent  seem  to  describe  activity  related  to  climbing 
and  descending  aircraft  in  the  vicinity  of  major  airports. 
When  considering  these  kinds  of  flights,  it  is  easy  to  see 
how  the  variables  that  describe  this  component  relate  to 


one  another.  For  example,  arrival  and  departure  traffic 
associated  with  Major  airports  (.68)  would  tend  to  in¬ 
crease  the  Number  of  climbing  and  descending  aircraft 
(.79).  Airspace  around  major  airports  tends  to  have  more 
Intersecting  flight  paths  (.63).  Traffic  volume  (.59)  also 
tends  to  be  higher  proximal  to  major  airports.  Increased 
traffic  directly  impacts  the  Amount  of  radio  frequency 
congestion  (.58)  and  in  some  sectors  would  increase  the 
Mix  of  aircraft  types  (.69).  The  Amount  of  coordination 
required  (.74),  Number  of  multiple  functions  (.73),  and 
Number  of  required  procedures  (.66)  represent  tasks  the 
controller  must  perform  in  complex  airspace  that  often 
surrounds  larger  airports  and  would  become  more  exi¬ 
gent  in  conjunction  with  the  other  factors.  Perhaps  the 
association  of  the  Size  of  sector  airspace  (.51)  represents  a 
relationship  between  the  size  of  the  sector  and  the  impact 
of  these  activities.  In  other  words,  their  effects  may  be 
mediated  by  the  amount  of  time  available  for  resolution 
or  completion. 

Component  2  had  an  eigenvalue  of  3 . 67  and  accounted 
for  approximately  17%  of  the  variance.  This  component 
comprises  complexity  issues  associated  with  low- altitude 
sectors  that  provide  approach  services  into  Non-towered 
airports  (.91),  the  variable  with  the  highest  loading  on 
this  component.  In  the  Indianapolis  airspace,  1 1  sectors 


Table  4.  Principal  Components  Analysis  Rotated  Component  Matrix 


Variable 

1 

Component 

2  3 

4 

Climbing  and  descending  traffic 

.79 

Mix  of  aircraft  types 

.69 

VFR  versus  IFR  traffic 

.90 

Number  of  intersecting  flight  paths 

.63 

Number  of  multiple  functions 

.73 

Traffic  volume 

.59 

.51 

Amount  of  military  or  other  special  traffic 

.76 

Number  of  required  procedures 

.66 

Coordination/  interfacing  required 

.74 

Number  of  major  airports 

.68 

Extent  operations  are  affected  by  weather 

.74 

Relative  frequency  of  complex  routings 

Special  Use  Areas 

.72 

Size  of  sector  airspace 

.51 

.56 

Longitudinal  spacing/  sequencing 

.54 

Adequacy  of  radio/radar  coverage 

.60 

Amount  of  radio  frequency  congestion 

.58 

Traffic  Management  Initiatives 

.59 

Terrain/Obstructions 

.88 

Shelves/Tunnels 

.53 

Foreign  aircraft/pilots 

.60 

Non-towered  airports 

.91 

*  Component  loadings  <  .50  not  shown. 
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provide  approach  services  to  airports  without  towers. 
Radar  coverage  does  not  always  reach  to  the  ground,  so 
a  loading  of  .60  for  the  Adequacy  of  radio/radar  coverage 
may  reflect  difficulties  associated  with  this  factor.  Aircraft 
flying  VFR  would  also  be  present  in  these  low- altitude 
sectors,  thus  increasing  the  ratio  of  VFR  versus  IFR 
traffic  (.90),  the  second-highest  loaded  variable.  The 
complexity  factor  Terrain  and  other  obstructions  (.88) 
is  exclusively  low- altitude  and  is  relevant  in  sectors  that 
provide  approach  services.  In  contrast,  sectors  with  non- 
towered  airports  do  not  have  a  high  mix  of  aircraft  types 
(as  most  aircraft  have  lower  performance  characteristics) , 
have  low  traffic  volume,  and  limited  frequency  congestion. 
Moreover,  these  sectors  have  limited  multiple  functions, 
coordination,  procedures,  complex  routings,  spacing  and 
sequencing,  traffic  management  initiatives,  and  shelves/ 
tunnels.  Thus,  it  makes  sense  that  complexity  ratings  for 
those  factors  did  not  load  on  Component  2. 

Component  3  had  an  eigenvalue  of 2. 78  and  accounted 
for  approximately  13%  of  the  variance  in  the  dataset. 
Variables  associated  with  this  component  are  primarily 
related  to  Military  operations  (.76)  and  other  Special 
Use  Area  (SUA)  restrictions  (.72).  In  the  ZID  airspace, 
tunnels  are  associated  with  military  operations .  This  might 
account  for  the  relationship  of  Shelves/Tunnels  (.53)  with 
this  dimension.  Restrictions  due  to  military  operations 
would  reduce  the  amount  of  usable  airspace.  Thus,  a 
.56  loading  of  Size  of  sector  airspace  on  this  dimension 
makes  sense.  However,  it  is  unclear  why  Foreign  aircraft/ 
pilots  would  be  associated  with  this  component  to  such 
a  high  degree  (.60).  Perhaps  the  unifying  theme  of  this 
dimension  is  that  each  of  these  factors  warrants  special 
consideration  or  attention,  and  Foreign  aircraft/pilots 
fall  into  this  category. 

Component  4  had  an  eigenvalue  of  2 . 3  5  and  accounted 
for  approximately  1 1  %  of  the  variance.  The  variables 
most  strongly  associated  with  this  component  relate  to 
difficulties  associated  with  inclement  weather,  as  evidenced 
by  the  two  highest-loaded  variables,  Extent  operations 
are  affected  by  weather  (.74)  and  Traffic  Management 
Initiatives  (.59).  Requirements  for  longitudinal  spacing/ 


sequencing  (.54)  and  problems  associated  with  Traffic 
volume  (.51)  are  also  magnified  by  inclement  weather. 

The  test  for  a  successful  PCA  is  simple:  A  good  PCA 
makes  sense.  That  certainly  appears  to  be  true  of  the 
components  extracted  from  the  Complexity-Q  ratings. 
In  this  particular  application,  the  component  scores 
can  be  used  to  empirically  test  some  of  the  assumptions 
made  during  component  interpretation.  For  example, 
if  Component  2  really  addresses  complexity  factors 
endemic  to  low- altitude  sectors  that  provide  services  to 
uncontrolled  airports,  then  Component  2  scores  should 
be  significantly  higher  in  those  sectors.  If  Component  3 
relates  to  problems  associated  with  military  airspace  and 
SUAs,  then  factor  scores  should  be  higher  in  sectors  that 
have  this  kind  of  airspace. 

To  gain  a  clearer  understanding  of  how  the  components 
relate  to  sectors  in  different  altitude  strata,  a  one-way 
analysis  of  variance  was  conducted  for  each  of  the 
components,  using  mean  component  scores  as  the 
dependent  variable.  The  results  demonstrated  that  mean 
component  scores  were  significantly  different  between  at 
least  two  of  the  sector  groups  for  Component  1  fF(2,34)  = 
7.85, p  <  .01],  Component  2  (A(2,34)  =  27.87, p  <  .01], 
and  Component  4  fF(2,34)  =  6.57,  p<  .01]. 

When  viewing  the  results  from  post  hoc  comparisons 
(Tables  5-7),  keep  in  mind  that  component  scores  are 
composite  variables  calculated  from  individual  responses 
weighted  by  the  component  loadings.  Consequently,  the 
means  no  longer  resemble  the  original  100-point  scale 
ratings.  Post  hoc  tests  revealed  that  high- altitude  sectors 
had  significantly  greater  mean  Component  1  scores  than 
super  high- altitude  sectors  (see  Table  5).  Although  mean 
Component  1  scores  appeared  to  be  greater  for  high- 
altitude  vs.  low-altitude  sectors,  this  difference  was  not 
statistically  significant. 

As  anticipated,  low-altitude  sectors  had  significantly 
higher  mean  Component  2  scores  than  other  types  of 
sectors  (Table  6).  The  mean  of  Component  2  scores  for 
all  low- altitude  sectors  was  .93.  However,  the  complexity 
factors  most  closely  associated  with  Component  2  better 
describe  low-altitude  sectors  that  provide  services  to 


Table  5.  Component  Score  Descriptive  Statistics  and  Post  hoc  Comparisons f: 
Component  1  by  Sector  Altitude  Strata 


Strata 

Low  High 

Mean  SD 

Low 

High 

Super  High 

p  =.05 

p  =.21  p  =.00 

-.08  .61 

.46  .51 

-.50  .62 

Uukey  HSD 


Table  6.  Component  Score  Descriptive  Statistics  and  Post  hoc  Comparisons': 
Component  2  by  Sector  Altitude  Strata 


Strata 

Low  High 

Mean  SD 

Low 

High 

Super  High 

p  =.00 

p  =.00  p  =.93 

.93  .93 

-.54  .23 

-.62  .15 

tTukey  HSD 


Table  7.  Component  Score  Descriptive  Statistics  and  Post  hoc  Comparisons': 
Component  4  by  Sector  Altitude  Strata 


Strata 

Low  High 

Mean  SD 

Low 

High 

Super  High 

p  =.02 

p=.01  p  =.81 

-.54  .80 

.11  .50 

.27  .32 

fTukey  HSD 


non-towered  airports.  Mean  Component  2  scores  in 
these  sectors  (1.41)  were  considerably  higher  than  mean 
Component  2  scores  in  low- altitude  sectors  that  do  not 
provide  such  services  (-.21).  Low- altitude  sectors  had 
significantly  lower  Component  4  scores  than  high-  and 
super  high-altitude  sectors.  This  suggests  that  difficulties 
related  to  inclement  weather  were  rated  lower  in  low- 
altitude  sectors  than  in  the  higher  altitudes.  As  with 
Component  2,  there  were  no  significant  differences 
between  high-  and  super  high-altitude  sectors  for 
Component  4  scores  (Table  7). 

Component  3  scores  did  not  vary  significantly  between 
sectors  with  different  altitude  stratum.  This  is  not 
surprising,  because  the  variables  most  strongly  associated 
with  this  component  (e.g.,  special  routes,  restricted  areas) 
are  found  at  all  altitudes.  On  the  other  hand,  Component 
3  scores  should  be  higher  in  sectors  subject  to  these  kinds 
of  restrictions.  Based  on  information  derived  from  sector 
descriptions  in  the  SOPs,  sectors  with  restricted  areas, 
military  training  routes,  aerial  refueling  routes,  and/or 
MOAs  were  grouped  for  comparison  with  other  sectors. 
Of  the  37  sectors  sampled,  26  met  one  of  more  of  these 
criteria.  Mean  Component  3  scores  were  slightly  higher  in 
these  sectors  (.02)  than  in  others  (-.09),  but  this  difference 
was  not  significant;  t( 35)  =  -6.12 ,  p  =  .55. 

Comparison  ofPCA  Results  with  Rodgers,  Mogford,  and 
Mogford  (1998) 

Although  differences  in  measures  and  methodologies 
between  the  present  study  and  Rodgers  etal.  (1998)  render 
direct  comparisons  problematic,  there  were  similarities 
between  some  of  the  components  that  are  worthy  of  note. 


Table  8  contains  loadings  for  the  first  and  third  extracted 
components  from  both  studies.  The  correspondence 
between  these  components  is  readily  apparent,  particularly 
with  regard  to  Component  1.  Rodgers  and  coworkers 
named  their  first  extracted  component  Traffic  Activity 
because  the  variables  associated  with  it  “appeared  to  be 
related  to  traffic  volume  and  activities  associated  with 
managing  aircraft”  (p.  13).  For  the  same  reason,  the  name 
might  accurately  describe  our  Component  1 . 

Size  of  sector  airspace  was  positively  associated  with 
both  Component  1  and  Component  3  in  our  model  but 
had  only  a  small  negative  loading  on  Component  1  and  a 
small  positive  loading  on  Component  3  in  Rodgers  et  al. 
(1998).  In  their  analysis,  Size  ofsector  airspace  was  strongly 
associated  with  Component  2  (not  shown) .  This  is  hardly 
surprising  as  their  Component  2  was  named  “Size”  and 
consisted  almost  entirely  of  static  sector  characteristics 
excluded  from  our  analysis.  By  the  same  token,  none  of 
the  components  extracted  in  the  Rodgers  et  al.  analysis 
could  have  corresponded  with  our  “Non-towered  airport” 
Component  2  because  they  did  not  analyze  Non-towered 
airports,  VFR  versus  IFR  traffic,  or  Terrain  and  other 
obstructions  as  complexity  factors. 

The  third  extracted  component  in  both  analyses  was 
related  to  Military  operations  and  other  Special  Use  Area 
restrictions.  In  the  Rodgers  etal.  (1998)  study,  Radio  and 
radar  coverage  had  a  prominent  negative  loading,  which 
was  inconsistent  with  our  findings.  While  recognizing 
that  this  variable  “did  not  seem  to  be  directly  related  to 
military  functions,”  they  suggested  that  it  might  have  been 
“conceptually  associated  with  military  airspace  [because] 
areas  that  have  poor  radio  and  radar  coverage,  or  are 
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Table  8.  Comparative  Component  1  and  Component  3  Loadings 


Variable 

Component  1 

Component  3 

Current 

study 

Rodgers  et  al. 
(1998) 

Current 

study 

Rodgers  et  al. 
(1998) 

Climbing  and  descending  traffic 

.79 

.84 

.11 

-.10 

Mix  of  aircraft  types 

.69 

.18 

.31 

.07 

VFR  versus  IFR  traffic 

.01 

NA 

.09 

NA 

Number  of  intersecting  flight  paths 

.63 

.45 

.08 

.34 

Number  of  multiple  functions 

.73 

.83 

.18 

.30 

Traffic  volume 

.59 

.63 

-.15 

-.14 

Amount  of  military  or  other  special  traffic 

.11 

.13 

.76 

.68 

Number  of  required  procedures 

.66 

.85 

.14 

.17 

Coordination/  interfacing  required 

.74 

.48 

.20 

.33 

Number  of  major  airports  (Hubbing) 

.68 

.55 

.11 

-.38 

Extent  operations  are  affected  by  weather 

.18 

.66 

.21 

.11 

Relative  frequency  of  complex  routings 

.26 

.55 

.35 

.06 

Special  Use  Areas 

.20 

.10 

.72 

.48 

Size  of  sector  airspace 

.51 

-.29 

.56 

.24 

Longitudinal  spacing/  sequencing 

.43 

.57 

-.02 

-.32 

Adequacy  of  radio/radar  coverage 

.14 

.12 

.45 

-.59 

Amount  of  radio  frequency  congestion 

.58 

.62 

.32 

-.09 

Traffic  Management  Initiatives 

.33 

NA 

.08 

NA 

T  errain/Obstructions 

-.02 

NA 

.11 

NA 

Shelves/Tunnels 

.21 

.09 

.53 

.44 

Foreign  aircraft/pilots 

.08 

NA 

.60 

NA 

Non-towered  airports 

.04 

NA 

.02 

NA 

controlled  by  the  military,  are  relatively  inaccessible  to 
FAA  ATC”  (p.  13).  This  explanation  could  just  as  easily 
apply  to  our  positive  loading  of  .45. 

In  our  analysis,  Relative  frequency  of  complex  routings 
was  not  associated  with  any  one  component,  but  in  the 
Rodgers  et  al.  ( 1 998)  study,  it  had  a  definitive  Component 
1  loading  (.55).  This  probably  reflects  disparate  sample 
characteristics.  The  only  discernable  divergence  between 
ratings  provided  by  supervisors  and  controllers  in  our 
sample  was  that  supervisors  consistently  rated  the  Relative 
frequency  of  complex  routings  higher  than  controllers 
did.  Unfortunately,  we  cannot  make  any  inferences  about 
the  comparative  influence  of  the  Relative  frequency  of 
complex  routings  in  ZID  and  ZTL  because  of  these 
sample  differences. 

In  spite  of  the  dissimilarities  noted,  it  is  the 
similarities  that  are  most  striking.  Considering  statistical 
differences  (i.e.,  rotated  vs.  un-rotated  matrix,  four  vs. 
six  extracted  components),  sample  differences  (i.e., 
ratings  provided  predominantly  by  controllers  vs.  ratings 
only  provided  by  supervisors),  variable  set  differences 
(i.e.,  22  complexity  factors  vs.  16  complexity  factors 
with  variables  derived  from  other  sources) ,  and  airspace 
differences  (i.e.,  ZID  vs.  ZTL),  the  obvious  parallels 


between  Components  1  and  3  and  the  dimensions  they 
describe  are  remarkable. 

Multiple  Regression  Analysis  of  Complexity  Factor  Scores 
With  Operational  Errors 

As  previously  mentioned,  one  of  the  primary  benefits 
of  using  PCA  in  this  application  is  consolidation  of  the 
individual  complexity  factors  into  a  reduced  number  of 
components.  The  method  used  to  compute  component 
scores  (i.e.,  weighting  individual  responses  according  to 
that  variable’s  relationship  with  the  dimension)  tends  to 
produce  variables  with  normal  distributions.  Moreover, 
orthogonal  rotation  methods  make  it  virtually  impossible 
for  the  components,  as  a  predictor  set,  to  suffer  from 
multicollinearity.  Therefore,  there  is  little  question  as  to 
their  appropriateness  for  multiple  regression  analysis.  In 
this  sample,  the  distribution  of  the  number  of  OEs  per 
sector  had  a  mean  of  6.68,  with  a  standard  deviation 
of  4.23  (Skewness  =  .61,  SE  Skewness  =  .388;  Kurtosis 
=  -.67,  SE  Kurtosis  =  .759).  This  was  less  than  two 
standard  deviations  from  normal  in  skewness  and  less 
than  one  standard  deviation  from  normal  in  kurtosis. 
Consequently,  both  the  predictors  (the  component  scores) 
and  the  criterion  (the  number  of  OEs)  met  assumptions 
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Table  9.  Multiple  Regression  Analysis:  Complexity  Component  Scores  on  Number  of 
Operational  Errors  (N  =  37) 


Variable 

B 

SEB 

t 

P 

Component  1 

4.62 

.69 

6.67 

.75** 

Component  2 

-1.18 

.51 

-2.33 

-.26* 

Component  3 

1.54 

.95 

1.62 

.19 

Component  4 

-.33 

.73 

-.45 

-.05 

p  =  <.01;  *p  =  <.05 


of  normality.  No  univariate  or  multivariate  outliers  were 
detected.  Studentized1  residuals  plotted  against  predicted 
values  were  randomly  distributed  in  a  horizontal  band 
around  zero,  indicating  that  the  assumption  of  linearity 
and  the  assumption  of  equality  ofvariance  were  met.  Visual 
examinations  of  the  cumulative  probability  plot  of  the 
observed  distribution  of  residuals  against  that  expected  of 
a  normal  distribution  demonstrated  that  the  assumption 
of  normally  distributed  errors  was  also  met. 

Standard  multiple  regression  of  the  extracted 
complexity  components  on  the  number  of  OEs  per 
sector  produced  a  multiple  R  =  .78  (R2  =  .61)  that  was 
significantly  different  from  zero,  F(4, 32)  =12. 72, p<. 01. 
Note  that  in  this  analysis  the  number  of  ZID  sectors  has 
been  reduced  from  40  to  37  due  to  sector  combinations 
that  were  recommended  by  ZID  personnel  to  facilitate 
the  administration  of  the  Complexity-Q  questionnaire. 
As  shown  in  Table  9,  Components  1  and  2  contributed  a 
significant  amount  of  unique  information  to  the  model, 
whereas  Components  3  and  4  did  not. 

Conclusions 

The  present  study  is  the  second  phase  of  a  multi-year 
project  (called  SCOpE)  to  better  understand  the  contri¬ 
bution  of  sector  complexity  measures  to  the  occurrence 
of  OEs  in  en  route  air  traffic  control.  To  do  this,  we 
plan  to  measure  complexity  by  examining  both  static 
and  dynamic  measures  as  well  as  subjective  and  objec¬ 
tive  measures.  The  static/ dynamic  distinction  involves 
relatively  unchanging  characteristics  of  the  sector  (such 
as  size,  strata,  length  of  jet  ways  and  airways,  number  of 
airports,  required  procedures,  number  of  shelves)  and 
more  transitory  aspects  of  the  traffic  situation  (such  as 
amount  of  traffic,  weather,  aircraft  mix,  climbing  and 
descending  aircraft).  The  subjective/objective  method 
incorporates  both  controllers’  opinions  about  the  impor¬ 
tance  of  complexity  factors  and  impartial  information 
extracted  from  environmental  data. 

1  Studentized  residual:  The  residual  divided  by  an  estimate  of  its 

standard  deviation  that  varies  from  case  to  case,  depending  on  the 
distance  between  the  case  value  of  the  independent  variable  and  the 
mean  of  the  independent  variable  (Norusis,  1990). 


Because  limited  data  were  available  during  the  first  year 
of  the  project  and  because  of  the  lengthy  processing  time 
required  to  produce  the  objective  dynamic  data  obtained 
during  the  second  year,  we  have  not  yet  been  able  to  evalu¬ 
ate  the  effectiveness  of  the  two  measurement  methods. 
In  this  study,  we  looked  at  subjective  ratings  for  a  set  of 
combined  static  and  dynamic  sector  characteristics.  Next 
year,  we  will  complete  the  project  by  adding  objective 
dynamic  sector  information  to  the  analysis. 

In  the  first  two  years  of  the  SCOpE  project,  we  pro¬ 
gressively  analyzed  the  relationship  between  objectively 
measured  static  sector  characteristics  and  OE  occurrence  in 
isolation.  This  does  not  mean  that  we  believe  they  function 
in  isolation.  As  Buckley  et  al.  (1983)  concluded,  traffic 
density  and  sector  geometry  “interact  in  a  complex  way” 
(p.  73).  The  modest  prediction  (43%)  of  OE  occurrence 
produced  by  the  static  variables  sector  size,  sector  strata, 
and  number  of  major  airports  in  the  Goldman  et  al. 
(2006)  study  was  to  be  expected  because  the  correspond¬ 
ing  objective  dynamic  variables  with  which  they  interact 
were  missing  from  the  equation.  However,  the  analyses  of 
static  characteristics  provided  us  with  a  baseline  by  which 
to  evaluate  the  objective  dynamic  measures. 

The  controllers’  subjective  ratings  provided  a  per¬ 
spective  about  the  importance  of  complexity  factors  not 
available  from  the  objective  data.  Furthermore,  the  PCA 
of  subjective  complexity  ratings  was  extremely  beneficial 
because  the  extracted  components  provided  a  complexity 
“profile”  of  the  ZID  sectors  as  perceived  by  the  controllers 
who  rated  them.  Four  components  were  extracted  that 
described  different  dimensions  of  complexity  common 
to  ZID  sectors. 

We  interpreted  the  meaning  of  the  components  based 
on  the  loadings  of  the  individual  complexity  factors.  By 
examining  the  relationship  between  objective  measures 
and  the  subjectively  derived  component  scores,  we  were 
able  to  discern  at  least  one  dimension  that  was  reliably 
associated  with  a  particular  type  ofsector  (i.e. ,  Component 
2,  that  described  sectors  providing  services  to  non-towered 
airports).  Thus,  we  identified  specific  complexity  factors 
that  are  particularly  relevant  in  these  sectors. 
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Other  complexity  components  were  more  difficult 
to  describe.  Component  1  appeared  to  be  related  to 
activities  associated  with  the  control  of  climbing  and 
descending  aircraft  in  the  vicinity  of  major  airports,  and 
Component  3  seemed  to  be  related  to  military  operations 
and  special  use  airspace.  However,  we  were  unable  to 
verify  our  interpretations  by  comparing  the  component 
scores  with  external  objective  measures.  Nevertheless,  the 
concordance  of  our  Components  1  and  3  with  the  first 
and  third  components  in  Rodgers  et  al.  (1998)  suggests 
that  these  dimensions,  though  difficult  to  describe,  may 
be  common  to  more  than  one  facility.  The  number  of 
possible  predictors  in  the  scientific  literature  and  the  lack 
of  concordance  between  studies  make  the  discovery  of 
two  potentially  stable  dimensions  a  major  finding. 

We  then  conducted  a  multiple  regression  analysis 
that  used  the  four  complexity  components  to  predict 
OE  occurrence.  Only  Component  1  (major  airports) 
and  Component  2  (non-towered  airports)  explained  a 
significant  amount  of  the  variance  in  OEs  in  the  ZID 
sectors  ( R  =  .78,  R2  =  .61).  Component  1  was  positively 
associated  with  the  number  ofOEs  (i.e.,  higher  scores  were 
related  to  a  higher  number  of  OEs) ,  whereas  Component 
2  had  a  negative  relationship  (higher  scores  were  related  to 
fewer  OEs) .  Considering  that  the  component  scores  were 
based  on  subjective  ratings  of  sector  complexity  factors 
and  did  not  include  any  variables  explicitly  describing 
individual  behavioral  or  organizational  aspects  associ¬ 
ated  with  OEs,  61%  explained  variance  is  impressive. 
However,  components  were  constructed  from  “human- 
weighted”  complexity  factors,  and  so  it  is  possible  that 
the  human  element  was  not  entirely  missing  from  the 
equation.  The  relationship  between  Component  2  and 
the  incidence  of  OEs  reminds  us  that  sector  complexity 
does  not  always  produce  a  negative  outcome.  Indeed,  a 
certain  degree  or  type  of  complexity  may  be  related  to  a 
reduction  in  OEs. 

The  insight  gained  by  analysis  of  the  subjective  rat¬ 
ings  provided  by  ZID  personnel  (i.e.,  the  identification 
of  complexity  dimensions  and  the  variables  most  closely 
associated  with  them)  will  guide  our  efforts  as  we  move 
on  to  the  next  phase  of  this  project.  Certainly  complexity 
characteristics  associated  with  Components  1  and  2  will 
be  the  subject  of  close  scrutiny.  Of  course,  complexity 
factors  associated  with  the  other  dimensions  will  also 
be  of  interest.  The  fact  that  Component  3  failed  to 
contribute  significantly  to  the  prediction  of  OEs  in  this 
analysis  does  not  mean  that  military  airspace  or  SUAs 
don’t  make  a  sector  more  difficult  to  work  or  increase  the 
likelihood  of  an  OE.  It  simply  means  that  subjective  rat¬ 
ings  of  these  factors  failed  to  predict  OEs.  Similarly,  the 
inability  of  Component  4  to  contribute  significantly  to 


the  regression  model  may  reflect  the  intermittent  nature 
of  this  dynamic  event,  or  it  may  simply  be  an  artifact  of 
the  way  the  variable  was  measured.  In  other  words,  the 
presence  of  inclement  weather  might  be  highly  correlated 
with  the  occurrence  of  OEs  but  the  component  scores 
based  on  subjective  ratings  of  variables  associated  with 
inclement  weather  were  not. 

The  next  phase  of  the  SCOpE  project  involves  analyz¬ 
ing  objective  measures  that  correspond  to  the  subjective 
ratings  of  dynamic  complexity  factors  examined  in  the 
present  study.  While  static  complexity  factors  (measured 
both  objectively  and  subjectively)  have  so  far  had  limited 
value  in  providing  useful  information  about  OE  occur¬ 
rence,  objective  dynamic  measures  should  be  much  more 
descriptive  and  relevant. 

Practical  prediction  models  (linear  or  otherwise)  must 
eventually  be  calculated  from  objective  measures  because 
the  actual  characteristics  of  the  sectors  must  be  addressed 
when  developing  strategies  to  reduce  OEs.  Nevertheless, 
subj  ective  information  about  the  importance  of  complex¬ 
ity  factors  can  guide  the  choice  of  objective  measures  for 
analysis  and  may  also  be  used  to  weight  their  importance. 
A  potential  application  for  the  information  gained  from 
this  study  is  a  tool  that  could  be  used  by  airspace  designers 
to  evaluate  the  potential  OE  risk  associated  with  different 
design  concepts. 
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APPENDIX  A 


Principal  Components  Rotated  Component  Matrix 


Variable 

1 

Component 

2  3 

4 

Climbing  and  descending  traffic 

.79 

.02 

.11 

.11 

Mix  of  aircraft  types 

.69 

.22 

.31 

-.13 

VFR  versus  IFR  traffic 

.01 

.90 

.09 

-.17 

Number  of  intersecting  flight  paths 

.63 

-.40 

.08 

.14 

Number  of  multiple  functions 

.73 

.07 

.18 

.22 

Traffic  volume 

.59 

-.28 

-.15 

.51 

Amount  of  military  or  other  special  traffic 

.11 

.19 

.76 

.19 

Number  of  required  procedures 

.66 

-.08 

.14 

.37 

Coordination/  interfacing  required 

.74 

.13 

.20 

.12 

Number  of  major  airports 

.68 

-.12 

.11 

.32 

Extent  operations  are  affected  by  weather 

.18 

-.03 

.21 

.74 

Relative  frequency  of  complex  routings 

.26 

-.35 

.35 

.39 

Special  Use  Areas 

.20 

.10 

.72 

.20 

Size  of  sector  airspace 

.51 

-.20 

.56 

-.15 

Longitudinal  spacing/  sequencing 

.43 

-.32 

-.02 

.54 

Adequacy  of  radio/radar  coverage 

.14 

.60 

.45 

.06 

Amount  of  radio  frequency  congestion 

.58 

.08 

.32 

.15 

Traffic  Management  Initiatives 

.33 

-.47 

.08 

.59 

Terrain/Obstructions 

-.02 

.88 

.11 

-.09 

Shelves/Tunnels 

.21 

.16 

.53 

-.16 

Foreign  aircraft/pilots 

.08 

-.10 

.60 

.45 

Non-towered  airports 

.04 

.91 

.02 

-.16 

A-l 


